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THE  ORIGIN  OF  COMPLEXITY 
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Department  of  Electrical  Engineering  and  Computer  Sciences. 

University  of  California  at  Berkeley, 
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Nature  abounds  with  complex  patterns  and  structures  emerging  from  homogeneous 
media  operating  far  from  thermodynamic  equilibrium.  Such  phenomena,  which  are 
widely  observed  in  both  inanimate  (non-biological)  and  biological  media,  can  be 
modeled  and  studied  via  the  CNN  (Cellular  Neural /Nonlinear  Network)  paradigm 
in  an  in-depth  and  unified  way.  Whether  a homogeneous  medium  is  capable  of 
exhibiting  complexity  depends  on  whether  the  CNN  cells,  or  its  couplings,  is  lo- 
cally active  in  a precise  mathematical  sense.  This  local  activity  principle  is  of 
universal  generality  and  is  responsible  for  all  symmetry  breaking  phenomena  ob- 
served in  a great  variety  of  non-equilibrium  media  ranging  from  the  emergence  of 
negative  differential  conductance  in  bulk  semiconductor  materials  (e.g.,  Gallium 
Arsenide  in  Gunn  Diodes)  to  the  emergence  of  artificial  life  itself.  The  main  re- 
sult of  this  paper  consists  of  a set  of  explicit  analytical  conditions  for  calculating 
the  parameter  ranges  necessary  for  the  emergence  of  a non-homogeneous  static  or 
dynamic  pattern  in  a homogeneous  medium  operating  under  an  influx  of  energy 
and/or  matter.  The  resulting  “complexity  related”  inequalities  are  applicable  to 
all  media,  continuous  or  discrete,  which  have  been  mapped  into  a CNN  paradigm. 


One  of  the  most  interesting  aspects  of  the  world  is  that  it  can  be 
considered  to  be  made  up  of  patterns.  A pattern  is  essentially 
an  arrangement.  It  is  characterized  by  the  order  of  the  dements 
of  which  it  is  made,  rather  than  by  the  intrinsic  nature  of  these 
elements. 

Norbert  Wiener 


1 Introduction 

How  does  the  leopard  get  its  spots?  How  does  the  zebra  get  its  stripes?  How  does 
the  fingerprint  get  its  patterns?  How  does  an  ant  colony  manage  to  self-organize 
into  an  impressive  pattern  of  activities  when  individual  ants  are  known  to  be  quite 
dumb?  How  does  an  initially  mixed  distribution  of  black  and  white  population  in 
a housing  community  manage  to  redistribute  over  time  into  segregated  black  and 
white  neighborhoods  with  well-defined  boundaries? 

How  does  a bee  sting  at  a finger  tip  trigger  the  propagation  of  an  electrical 
impulse  to  the  brain  of  a healthy  person,  and  how  does  this  distress  signal  fail 
to  propagate  in  the  nerves  of  patients  suffering  from  multiple  sclerosis?  How  do 
some  members  of  a colony  of  starving  amoeba  send  out  a target  and  spiral  wave 
signal  to  attract  neighboring  amoebae  and  transform  them  into  spores,  and  then  to 
regenerate  into  amoeba  again  when  food  (bacteria)  becomes  available?  How  does  a 
“scroll  wave”  get  generated  in  the  cardiac  muscle  by  the  inadvertent  presence  of  an 
electrical  impulse  during  a vulnerable  window  of  a few  milliseconds,  often  leading 
to  sudden  cardiac  death? 
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The  above  phenomena  are  some  manifestations  of  a multidisciplinary  paradigm 
called  emergence,  or  complexity.  They  share  a common  unifying  principle  character- 
istic of  dynamic  arrays , such  as  cellular  neural  networks , namely,  interconnections  of 
a sufficiently  large  numbers  of  simple  dynamical  units,  which  can  exhibit  extremely 
complex,  synergetic,  and  self-organizing  behaviors.  The  common  denominator  in 
all  of  these  pattern  formation  and  active  wave  propagation  phenomena  is  the  pres- 
ence of  an  active  medium,  powered  by  a constant  supply  of  external  energy.  For 
example,  in  the  brain,  the  active  medium  is  provided  by  a sheet-like  array  of  mas- 
sively interconnected  excitable  neurons  whose  energy  comes  from  the  burning  of 
glucose  with  oxygen.  In  cellular  neural  networks,  the  active  medium  is  provided 
by  the  local  interconnections  of  active  cells,  whose  building  blocks  include  active 
nonlinear  devices  (e.g.,  CMOS  transistors)  powered  by  batteries. 

Research  on  Emergence  and  Complexity  has  gained  immense  momentum  during 
the  past  decade1-15.  The  fundamental  problem  is  to  uncover  nature’s  secret  mech- 
anisms which  are  responsible  for  the  self  organization  and  spontaneous  emergence 
of  many  stable  complex  ( static  and  dynamic ) patterns0  in  homogeneous  media  op- 
erating far-from-thermodynamic  equilibrium  16 . Indeed,  nature  is  abound  with  all 
sorts  of  patterns  ranging  from  regular  snow  flakes  to  chaotic  brain  waves17,18,19. 
Understanding  and  controlling  such  patterns  is  essential  for  designing  new  genera- 
tions of  brain-like  molecular  devices  and  systems  endowed  with  artificial  intelligence 
and  self-repair  capabilities. 

The  homogeneous  media  alluded  to  above  consists  usually  of  an  active  bulk 
medium  (e.g.,  bulk  materials  with  negative  resistivity,  such  as  Gallium  Arsenide  in 
Gunn  Diodes20,  nerve  membranes,  heart  tissue  layers,  chemical  mixtures  in  stirred 
reactor  tanks,  etc.)  modeled  by  one  or  more  nonlinear  PDE ’s  (partial  differential 
equations)  where  the  spatial  coordinate,  as  well  as  the  state  variables,  are  repre- 
sented by  continuous  real  numbers.  What  is  truly  fascinating  is  that  while  these 
active  media  are  completely  unrelated — they  can  range  from  inanimate  materials 
to  living  tissues21 — yet  the  patterns  they  exhibit  tend  to  resemble  each  other  under 
appropriate  initial  and  boundary  conditions.  It  makes  sense  therefore  to  hypothe- 
size that  a common  mechanism  must  be  responsible  for  the  emergence  of  each  type 
of  patterns  (e.g.,  Turing  patterns,  spiral  waves,  etc.)5.  This  remarkable  observa- 
tion motivates  the  development  of  a unified  paradigm  capable  of  exhibiting  most,  if 
not  all,  static  and  dynamic  patterns  (i.e.,  dissipative  structures)  in  active  homoge- 
neous media  operating  far  from  thermodynamic  equilibrium.  Such  a paradigm  has 
been  developed  recently  and  is  the  subject  of  a recent  treatise  22 . The  paradigm  is 
dubbed  the  CNN,  an  acronym  for  cellular  neural  networks  when  used  in  the  context 
of  brain  science,  or  cellular  nonlinear  networks,  when  used  in  other  more  general 
contexts. 

A CNN  is  defined  by  two  mathematical  constructs: 

1.  A spatially  discrete  collection  of  nonlinear  dynamical  systems  called  cells, 
where  information  can  be  encrypted  into  each  cell  via  3 independent  variables 


“Such  patterns  are  called  dissipative  structures  by  Ilya  Prigogine14  because  energy  must  be  con- 
tinually  supplied  and  dissipated  in  order  to  maintain  such  structures. 

feFor  related  works  on  emergence  and  complexity  from  different  perspectives,  the  reader  is  referred 
to  the  very  readable  expositions  by  Crick8,  Eigen10,  Gell-Mann7,  and  Prigogine14. 
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Figure  1.  A CNN  with  a cutout  view  which  exposes  an  inner  cel)  Cijk  and  its  sphere  of  influence 
Sijfc.  Only  those  cells  Cap^  located  within  are  coupled  to  cell  Cy*. . 


called  input,  threshold,  and  initial  state. 

2.  An  interconnection  law  relating  one  or  more  relevant  variables  of  each  cell 
Cij  to  all  neighbor  cells  Cki  located  within  a prescribed  sphere  of  influence  Sij  (r) 
of  radius  r,  centered  at  Cy. 

In  the  special  case  where  the  CNN  consists  of  a homogeneous  array,  and  where 
its  cells  have  no  inputs,  no  thresholds,  and  no  outputs,  and  where  the  sphere  of 
influence  extends  only  to  the  nearest  neighbors  (i.e.,  r = 1),  the  CNN  reduces  to 
the  familiar  concept  of  a lattice  dynamical  system  from  mathematics. 

The  schematic  diagram  of  a 3-dimensional  CNN  is  shown  in  Fig.l,  where  a 
typical  cell  C^k  is  highlighted  along  with  its  sphere  of  influence  Sijk  ■ Let  us  consider 
some  examples. 

Example  1.  Emergence  of  3-Dimensional  Knot  Patterns 

Suppose  each  cell  in  Fig.l  consists  of  a Chua’s  circuit  with  three  external  ter- 
minals (one  of  them  being  the  ground  reference  terminal),  as  shown  in  Fig. 2.  Since 
each  cell  can  interact  with  its  neighbors  only  through  the  2 ungrounded  terminals, 
each  one  coupled  to  a corresponding  node  of  a neighbor  cell  via  a resistor,  each 
ungrounded  terminal  serves  as  a port  where  energy  can  flow  into  or  out  of  the  cell. 
Consequently,  the  3-terminal  circuit  cell  in  Fig. 3 is  also  called  a 2-port  in  electri- 
cal engineering.  Suppose  each  cell  in  Fig. 2 is  coupled  to  its  6 nearest  neighbor  cells 
(two  along  each  coordinate  axis)  via  positive  linear  resistances,  as  depicted  in  Fig. 2. 
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A section  of  the 
coupling  circuit 


Chua’s  Circuit 


Sphere  of  influence  Sijk  of  cell  C(ijk)  containing  all  cells  C(affy)  £ Sijk 


Figure  2.  Each  cell  in  this  3-dimensional  CNN  is  a Chua’s  oscillator  with  3 external  terminals, 
or  2 ports,  with  a common  ground.  Each  external  terminal  (except  the  ground),  or  port,  of  each 
cell  is  connected  to  a capacitor  inside  the  cell,  and  coupled  to  a corresponding  terminal  of  the  6 
neighboring  cells  via  6 linear  passive  resistors. 


If  all  elements  inside  the  cell  shown  in  the  inset  are  passive,  and  if  the  nonlinear 
resistor  is  not  locally  active  (e.g.,  a pn  junction  diode)  in  the  sense  of  Definition  1 
in  Sec. 4. 2,  then  it  can  be  shown  that  given  any  initial  voltage  distributions,  all 
node-to-datum  voltages  must  tend  to  zero  as  t - — > oo.  In  other  words,  this  CNN 
must  have  a homogeneous  (uniform)  solution  at  all  nodes.  This  homogeneity  is 
generally  expected  since  all  cells  in  Fig. 7,  and  their  couplings,  as  well  as  boundary 
conditions,  are  identical,  and  there  are  no  input  sources ,c 

cWe  assume  throughout  this  paper  that  the  CNN  has  no  input  sources  and  has  a symmetrical 
boundary  condition  (e.g.,  a zero-flux  or  Neumann  boundary  condition),  since  the  central  concept 
of  emergence  implies  that  any  non-homogeneous  output  pattern  must  emerge  via  self  organization, 
and  not  through  any  external  input,  or  non-symmetrical  boundary  conditions. 


(C) 


(d) 


Figure  3.  Three-dimensional  knot  voltage  distribution  patterns  from  a CNN  cube  made  of  Chua’s 
oscillator  cells,  coupled  via  a linear  passive  resistive  grid,  (a)  A Hopf-link  pattern,  (b)  A 3-knot 
pattern,  (c)  A simple  helix,  (d)  A double  helix. 


However,  if  the  nonlinear  resistor  is  chosen  to  be  a Chua’s  diode  23 , as  shown 
in  the  inset  in  Fig. 2,  then  for  the  fixed,  choice  of  circuit  parameters  22,  the  4 
distinctly  different  3-dimensional  structures  in  Fig. 3 can  be  obtained  in  steady 
state  by  choosing  the  4 different  sets  of  initial  conditions  22 . Note  that  this  CNN 
is  symmetrical  with  respect  to  the  center  of  the  CNN  cube  in  Fig.  2 and  there 
are  no  inputs.  Yet  we  have  a non-uniform  constant  steady  state  node-to-datum 
voltage  distribution  ! In  the  parlance  of  complexity  theory,  we  say  the  homogeneous 
CNN  undergoes  a symmetry  breaking  and  any  non-homogeneous  node-to-datum 
steady  state  voltage  distribution  is  called  a pattern,  or  a dissipative  structure  if 
the  medium  is  non-conservative  in  the  sense  that  energy  dissipation  is  essential  to 
maintain  the  structure. 
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Figure  4.  (a)  A straight  scroll  vortex,  (b)  A twisted  scroll  wave. 


Example  2.  Emergence  of  3-Dimensional  Scroll  Waves 

Consider  the  same  CNN  cube  as  in  Fig. 2 except  that  in  this  case,  only  the  port 
voltage  across  the  nonlinear  resistor(Chua’s  diode)  is  coupled  to  corresponding 
node  voltages  of  the  6 neighboring  cells.  This  is  equivalent  to  open  circuiting  the 
second  port  (on  the  right)  of  the  Chua’s  oscillator  in  Fig. 2,  so  that  each  cell  reduces 
to  a one-port  circuit  with  2 external  terminals  (including  the  ground  reference 
terminal).  Using  the  circuit  parameters,  and  the  initial  and  boundary  conditions  22 , 
we  obtain  the  two  scroll  wave  structures  shown  in  Figs. 4(a)  and  4(b),  respectively, 
for  one  instant  of  time.  Unlike  the  stationary  structures  shown  in  Fig. 3,  the  scroll 
waves  in  Fig. 4 represent  an  active  nonlinear  wave  which  evolves  continuously  with 
a scrolling  structure  at  all  times. 

Example  3.  Emergence  of  2-Dimensional  Spiral  Waves 

If  we  consider  a 2-dimensional  version  of  the  preceding  CNN  cube,  we  would 
obtain  the  simplified  2-dimensional  CNN  shown  in  Fig. 5,  where  each  “Chua’s 
oscillator” -one- port  is  represented  by  a 2-terminal  black  box,  coupled  to  its  neigh- 
bors by  4 linear  positive  resistances,  two  along  each  coordinate  axis.  Using  the 
circuit  parameters  and  the  initial  and  boundary  conditions  given  in  22 , we  obtain 
the  spiral  wave  structure  shown  in  Fig. 6.  Again,  this  is  a dynamic  pattern  which 
rotates  continuously  for  all  times. 
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Figure  5.  A two-dimensional  CNN  made  of  Chua’s  oscillator  cells  coupled  via  a single  layer  of 
linear  resistive  grid. 


2 Mapping  PDE  Into  CNN 

A vast  majority  of  active  homogeneous  media  which  are  known  to  exhibit  com- 
plexity in  the  form  of  dissipative  structures  are  modeled  by  a reaction  diffusion 
PDEU~18: 


C7X ' 

Qt  = fii.x li  ®2i  •••>  xn)  ~t~  Di 

i = 1,2,  • n 


d2Xi  d2Xj  d2Xi  \ 

dx2  dy 2 dz 2 / ’ 


(1) 


where  x = (xi,  X2, ...,  xn)T  are  state  variables,  (x,  y,  z)  are  spatial  coordinates, 
f(x)  = (/i(x),  /2(x), ...,  /n(x))  is  a nonlinear  vector  function  of  x called  the  kinetic 
term,  and  D\,  Dy,  ...,  Dn  are  constants  called  diffusion  coefficients.  Replacing  the 
Laplacian  in  Eq.(l)  by  its  discretized  version 


d2Xi  d2Xi 

dx 2 ^ dy 2 


+ 


d2xi 
' dz 2 


(V 


(2) 
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where 


V2(xa^i7)i  = Xi(a  + 1,  /?,  7)  7 Xi(a  - 1,  /3, 7)  7 Xj(a,  /?  7 1, 7)  + Xi(a,  /3  - 1, 7)  7 
x»(a,  /?,  7 + 1)  + Xi(a,  A 7 — 1)  — 6xj(a,  /?,  7) 

2 = 1,2,  ...,n  (3) 


is  the  discrete  Laplacian  operator  on  the  it/2  component  Xi  of  the  state  variable 
x = (x\,X2,  ...,xn)T  about  the  grid  point  with  spatial  coordinate  (a,  /?, 7).  We 
obtain  the  following  associated  Reaction  Diffusion  CNN  equation 


a* (a,  ft  7)  = ft  7),  z2(a!,  /?,  7), xn(a,  ft  7))  + AV2(xai/3i7)i  (4) 

where  i = 1,2,  ...,ra;a  = (1,2,  ...,Na),p~  (1,2,  ...,Np),  and  7 = (1,2,  ...,iV7). 

Here  Xj(a,  /?, 7)  denotes  the  state  variable  Xi  located  at  a point  in  the  3-dimensional 
space  with  spatial  coordinate  (a,/?, 7).  Observe  that  the  Reaction  Diffusion  CNN 
equation  (4)  consists  of  a system  of  AT  = nNaNpN 7 ordinary  differential  equations 
(ODE’s). 

We  will  henceforth  refer  to  the  process  of  transforming  a PDE  into  a CNN 
equation  as  mapping  a PDE  into  a CNN.  Table  1 shows  the  mapping  of  4 
well-known  reaction  diffusion  PDE’s. 


Table  1.  Mapping  Reaction- Diffusion  PDE  into  a Reaction  Diffusion  CNN. 


FitzHugh-Nagumo  CNN  Equation 


+Di  [ti»+i  + Ui-i  - 2 U{] 

Vi  = — e[ui  — bvi  7 a] 
Brusselator  CNN  Equation 


du 

dt 

dv 

dt 


FitzHugh-Nagumo  PDE 

■3  ' d2u 


= -[— -u)-v  + D 


dx2 


— — elu  — bv  7 a] 


Brusselator  PDE 


Uij  — a (b  7 l)i/jj  7 V’ijVij  -j-  D\  1 
~\~Ui~itj  7 V>i7j- f-l  T U{  j—\  4u ] 
Vij  — buij  7 D‘2  \yiJr\p 

+Vi-I,j  7 Vij+ 1 7 Vij- 1 - 4 Vij\ 


du 

dt 

dv 

dt 


(b  + l)u  + u2v  7 Di 


= bu  — u2v  + D2 


d2v 


d2u 
dx 2 
d2v  1 


dx2  dy 2 


d2u 

+ W_ 


Meinhardt-Gierer  CNN  Equation 
auT 

Uij  — pUi-j  7 D\ [ Vi-f-i p 7 j 

vij 

PV’ip- {-1  T Vi]j  — 1 

Vij  = au2j  - '•jVij  7 D2[vi+itj  7 «i-ij 
+Vi,j+i  7 ~ 4 Vij\ 


du 

dt 

dv 

dt 


Meinhardt-Gierer  PDE 


au 2 
v 


— ffu  + Di 


= au 2 — 'jv  7 D2 


d2u  d2u 
dx2  + dy2 
d2v  d2v 
dx 2 dy2 


Figure  6.  Three-dimensional  view  of  a steadily  rotating  spiral  wave  at  one  instant  of  time.  The 
vertical  axis  (perpendicular  to  the  plane)  represents  the  capacitor  voltage  across  Chua’s  diode  and 
the  horizontal  axes  (on  the  plane)  are  the  spatial  coordinates. 


Oregonator  CNN  Equation 
tUijk  — ‘U'ijk  Vijk  ^'U'ijk  ^ijk^ijk  "T 

1^1  [ui-\-l,j,k  H"  l,fc 

— l,fe  + k-\-l  + ui,j,k  — 1 

V ijk  — Vijk  H-  fiuJijk  Uij  kVijk  H- 

“1“  1 

-6t/yk] 

yjijk  ~ U>ijfc  UJijfc  + -O3  [^2+1  ,k 

Ijjjfe  "1“  Witj— itk 

H-lUi^fe+1  “h  1 ■ 


Oregonator  PDE 


du 

= U + V 

2 

€!ti  = 

— cm 

- UV 

+Di 

d2u 

d2u 

d2u 

|co 

dx2 

+ 

dy2 

+ Si2 

= -V  + 

(3w  - 

■ UV 

+-02 

d2v 

d2v 

d2v 

dx 2 

+ 

dy 2 

+ d^ 

5i/; 

dt 

= li  — w 

+D3 

32w 

+ 

32™ 

• d2w 

3a;2 

3y2 

+ dz 2 
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Reaction  diffusion  PDE’s  form  an  important,  albeit  relatively  small,  subset  of 
the  universe  of  all  nonlinear  PDE’s.  For  any  other  nonlinear  PDE,  the  spatial 
derivative  (of  any  order)  of  any  state  variable  Ui(t;x,y,z)  at  time  t and  spatial 
location  d ( x , y,  z)  € R3  can  be  approximated  to  any  desired  accuracy  24,25  by 
finite  differences  involving  only  the  values  of  the  state  variable  ufft;  a',  /?',  7')  lo- 
cated at  a finite  number  of  lattice  points  ( x,y,z ) = (o',/?', 7')  £ 5’a/g7,  where 
(a,  /3, 7)  G Z3  denotes  the  “discrete”  coordinates  of  a 3-dimensional  lattice  Z3,  and 
Sa8~,(r)  denotes  a neighborhood  of  radius  r,  centered  at  (o,/3, 7).  Hence,  given 
any  nonlinear  PDE,  we  can  generate  many  (depending  on  the  desired  accuracy) 
approximate  discretized  systems  of  ODE’s  in  terms  of  a 3-dimensional  array  of 
state  variables  xffa,  f3 , 7),  a — 1, 2, ...,  Na,  ft  = 1,2, ...,  Np,  7 = 1, 2,  ...,NT  More- 
over, since  all  finite- difference  operations  involve  only  variables  located  within  a 
local  neighborhood,  we  can  always  decompose e the  discretized  system  into  a compo- 
nent (the  isolated  cell)  which  involves  only  Ui(t\ a , /?,  7)  at  the  lattice  site  (a,  (3, 7), 
and  another  component(  the  cell  coupling)  which  involves  all  neighboring  cells 
(a',/?', 7')  € Sap~t(r).  In  other  words,  given  any  nonlinear  PDE,  we  can  induce 
many  associated  CNN  equations — the  examples  given  in  Table  1 represent  the  sim- 
plest examples.  Although  it  is  not  true  that  the  qualitative  behaviors  of  a nonlinear 
PDE  and  its  associated  CNN  equation  are  always  the  same — the  propagation  fail- 
ure phenomenon26  is  a case  in  point,  extensive  computer  experiments  have  shown 
that  for  the  vast  majority  of  cases,  the  respective  solutions  can  be  made  virtually 
indistinguishable  by  choosing  a sufficiently  large  array  size  and  by  optimizing  the 
CNN  cell  and  coupling  parameters  22 . 

It  is  important  to  observe  that  partial  differential  equations  are  merely  mathe- 
matical abstractions  of  nature.  The  concept  of  a continuum  is  in  fact  an  idealization 
of  reality.  Even  the  collection  of  all  electrons  in  a solid  does  not  form  a continuum 
because  much  of  the  volume  separating  the  electrons  from  the  nucleus  of  atoms-f 
represents  a vast  empty  space!  In  fact,  recent  works  by  Smolin  and  his  colleagues 
have  proved  that  “the  spectrum  of  the  volume  of  any  physical  region  is  discrete"27 . 
In  particular,  quantum  mechanics  implies  that  at  extremely  small  distances  even 
space  is  made  of  discrete  bits! 

3 Local  Activity  Is  the  Origin  of  Complexity 

Let  Af  be  a 2-dimensional  CNN  associated  with  a homogeneous  medium.  By  defini- 
tion, a CNN  is  defined  uniquely  by  cells  and  their  interactions.  Let  us  identify  each 
cell  C(j,  k)  as  a nonlinear  m-port  defined  by  its  cell  dynamics,  where  m is  equal  to 
the  number  of  state  variables  which  are  directly  coupled  to  its  neighbors,  as  depicted 
by  the  “to”  external  (ungrounded)  terminals  attached  to  each  cell  C(j,k ) in  Fig.  7 
for  a reaction  diffusion  CNN.  Note  that  the  cell  C(j,  k)  may  contain  additional  state 

dTo  avoid  clutter  we  restrict  our  discussion  to  the  3-dimensional  Euclidean  space  R3 . The  same 
formulation  is  valid  for  any  dimension. 

cIn  most  cases,  the  decomposition  consists  of  just  the  superposition  of  these  two  components. 
However,  more  complex  decompositions  (e.g.,  nonlinear  functional  compositions)  may  be  required 
for  some  nonlinear  PDE’s. 

^The  ratio  between  the  diameter  of  the  electron  orbits  in  an  atom  to  the  diameter  of  its  nucleus 
is  of  the  order  of  105. 
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variables  which  are  not  directly  coupled  to  its  neighbors  and  hence  are  suppressed 
in  Fig. 7 to  avoid  clutter.3 

Let  us  identify  next  the  couplings  from  cell  C(j,  k)  to  all  cells  within  the  3x3 
sphere  of  influence  Sjk  centered  at  (j,  k)  by  a 7- port  For  the  one-diffusion 

reaction  diffusion  CNN  shown  in  Fig. 5,  an  isolated  cell  C{j,k)  and  its  4 coupling 
resistors  are  shown  in  Fig. 8(a).  Note  that  the  5- terminal  coupling  circuit,  or  7- port 
Tjk  (7  = 4 in  this  case),  in  Fig. 8(a)  can  be  redrawn  into  the  form  of  a grounded  4- 
port  (i.e.,  with  a common  ground  terminal)  in  Fig. 8(b).  For  the  reaction-diffusion 
CNN  depicted  in  Fig. 7,  the  7-port  Fj*.  is  composed  of  m identical  grounded  4- 
ports(with  node  (j,  k ) as  the  ground  node)  made  of  4 identical  positive  resistances, 
so  that  7 = 4m.  Observe  that  a CNN  is  completely  specified  by  the  m-port  cell 
C(j,  k ) and  the  7 -port  coupling  Tjk  since  they  can  be  used  as  a template  to  build 
up  a CNN  of  any  array  size. 

Since  the  conductance  of  all  resistors  in  each  layer  “i”  of  the  resistive  grid  in 
Fig. 7 is  equal  to  the  diffusion  coefficient  Di,  which  is  assumed  to  be  positive,  it 
follows  that  the  7-port  coupling  F^  is  passive.  If  the  cell  C(j,k)  is  not  locally 
active,  then  it  follows  from  symmetry  considerations  and  the  qualitative  theory  of 
nonlinear  networks28,29,30,31  that  the  CNN  must  have  a unique  steady  state  solution, 
thereby  implying  that  all  nodes  belonging  to  the  same  layer  must  have  identical 
node-to-datum  voltages.  It  follows  that  no  patterns  or  dissipative  structures  can 
exist. 

In  the  general  case,  the  7-port  coupling  Tjk  may  consist  of  a nonlinear  dy- 
namical multi-port.  In  this  case,  the  dynamics  must  again  tend  to  a homogeneous 
node  voltage  distribution  on  each  resistor  grid  if  both  the  m-port  cell  C(j,  k)  and 
the  F-port  coupling  F^  are  not  locally  active.  The  above  analysis  justifies  the 
following  fundamental  result  on  complexity: 

The  Local  Activity  Principle 


A CNN  associated  with  a homogeneous  non-conservative  (i.e.,  not  lossless)  medium 
having  a zero- flux  boundary  condition  can  not  exhibit  patterns  or  dissipative  struc- 
tures unless  the  cells,  or  the  couplings,  are  locally  active. 


4 Local  Activity  for  Reaction-Diffusion  CNNs 

In  general,  each  cell  C(j,  k,  l)  in  a reaction-diffusion  CNN  has  n state  variables  but 
only  m < n among  them  are  coupled  directly  to  its  nearest  neighbors  via  diffusion. 


9 A state  variable  x is  suppressed  if  its  associated  diffusion  coefficient  Dk  is  zero. 
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In  this  case,  the  state  equation  of  each  cell  C(j,  A,  Z)  assumes  the  form 

Vi(3,  k,  l ) = /i(Vi(j,  k,  Z),  V2(j,  k,  Z), ....  Vn(j,  k,  0)  + AV2Vi(j,  A,  l ) 

V2(j,  k,  l ) = /2(^(j,  A,  0,  V2(j,  A,  Z), F„(i,  A,  0)  + D2S72V2(j,  k,  l ) 

Vm(j,  A,  Z)  = k,  Z),  V2(j,  A,  Z), K(i,  A,  Z))  + DmV2Vm(j,  A,  Z) 

> (5) 

Kn+i(i,  A,  Z)  = fm+i  (Vi  (i,  A,  Z),  V2(j,  A,  Z), 14  (j,  A,  Z)) 

14  (j,  A,  Z)  - /«(Vi(j,  A,  Z),  V2(j,  A,  Z), ....  I4(j,  A,  Z)) 

j — 1,2,  ...,Nx',k  — 1,2,...,  ZVj,;  Z = 1,2,  ...,NZ. 

Alternatively,  we  can  include  the  remaining  Laplacian  terms  £>m+i  V2V4+i(j,  A,  Z), 
Hm+2 V2Kn+2(j,  A,  Z),  ...,  DnS72Vn(j,k,l)  in  Eq.(5)  and  set  £>m+i  = £>m+2  = ...  = 


Dn  = 0.  Recasting  Eq.(5)  into  vector  form,  we  obtain 

Va  =fa(Va,Vfc)  + DaV2Va  (6) 

Vb  =fb(Va,Vb)  (7) 

where 

Va  - [Vi(i,  A,  Z),  V2(j,  A,  Z),  A,  Z)]T  (8) 

Vb  = [Vm+1(j,  A,  Z),  Em+2(i,  A,  Z),  ...V4(j,  A,  Z)]T  (9) 

fa  = [/i(-),/2(0.  -/-(•)]  (10) 


fb  — [/m+l(')i  /m+2(')>  ••■/«(■)] 


(11) 


V2Va  = [V2Vi(j,  A,  Z),  V2E2(j,  A,  Z), ....  VVm(j,  A,  Z)] 


(12) 


Da 


£>1 


£>2 


£>„ 


, A > 0 


(13) 


To  emphasize  that  a CNN  is  defined  by  specifying  the  dynamics  of  the  cells  and 
their  couplings  ( cell  interaction  laws),  let  us  rewrite  Eqs.(6)-(7)  into  the  following 
standard  form  32 : 


cell  dynamics: 


Va  = fa(Va,Vb)+Ia 

(14) 

Vb  = fb(Va,Vb) 

(15) 
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cell  interaction  laws: 

la  = g a (v„(j,  k,  l ),  Va(j  - 1,  k,  Z),  Va(j  + 1,  *,  i),  VB(j,  * - 1,  0.  V„(j,  fc  + 1,1), 


V0(j1fclZ-l),V„(jIfc,I  + l)) 


'T>i 

d2 

V2v2(j,k,l) 

Dm 

y2vm(j,k,i)_ 

It  is  important  to  observe  that  Eqs.(14)-(15)  defining  the  cell  dynamics  involve  only 
the  voltage  variables  (V0(j,  k,  l),  Yb(j > k,  l))  and  the  current  variables  I a(j,  k,  l)  at 
the  same  spatial  location  (j,  k,  l),h  whereas  Eq.(16)  defining  the  cell  interaction  laws 
involves  not  only  the  voltage  variable  Va(j,  k,  l ) at  spatial  location  (j,  k,  l ),  but  also 
those  of  the  neighboring  cells.  The  relationship  between  a typical  cell  C(j,k,l) 
at  location  (j,  k , l ) and  its  coupling  network  is  shown  in  Fig. 7 for  a 2-dimensional 
CNN  reaction  diffusion  equation,  where  Va  = [Vi (j,  k),  V2(j,  k), Vm(j,  fc)]T  and 
Ia  = [Ii(j,  k),  I2 (j,  k), ...,  Im(j,  /c)]T  are  the  “m”  port  voltage  and  port  current  vari- 
ables. In  the  special  case  when  m = 1,  Fig. 7 reduces  to  Fig. 8(a),  where  cell  C(j , k) 
degenerates  into  a one-port. 


4-1  Cell  Equilibrium  Points 

Let  us  define  the  static  characteristic  of  an  isolated  CNN  cell  by  setting  Va  = 0 
and  Vs  = 0 in  Eqs.(14)  and  (15);  namely, 


fa(Vo,Vfc)  + Ia  = 0 

(17) 

ft(va,v6)  = o 

(18) 

where  Va  G Rm,  Ia  G Rm,  Vb  G Rnm,  fQ  G Rm,  fb  G R”~m.  Solving  Eq.(18)  for 
V;,  in  terms  of  Va,  we  obtain 


vfe  = ga(Vtt)  (19) 

where  ga(  ) may  be  a multi-valued  function  of  V0. 

Substituting  Eq.(19)  for  Yb  in  Eq.17,  we  obtain  the  following  implicit  and 
possibly  multi-valued  function: 


Static  Cell 
Characteristic 


0(V„,  Ia)  = fa (Va , ga(Va))  + Ia  - 0 (20) 


hThis  observation  allows  us  to  suppress  the  spatial  coordinates  in  Eqs.(14)-(15)  without  ambiguity. 
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Let  us  solve  Eq.(20)  for  Va.  In  general,  there  can  be  many  solutions  Va  — V„(Q] ), 
Va(<32),  • ••,  Va(<5P)  for  each  Ia  = Ia  € Rm,  where  Va(Qi)  denotes  the  port 
voltage  solution  Va  at  the  ith  solution  Qi.  We  call  each  solution  Vtt  = Va(Qj) 
a cell  equilibrium  point  associated  with  I„  = IQ  £ Rm.  In  other  words,  each  cell 
equilibrium  point  Va(Qi)  is  parameterized  by  Ia  6 Rm.  The  loci  of  all  such  cell 
equilibrium  points  calculated  explicitly  at  Ia  = Ia , as  Ia  ranges  over  the  entire  Tri- 
dimensional Euclidean  space  Rm  is  identical  to  the  static  cell  characteristic  defined 
implicitly  in  Eq.(20). 


4-2  Cell  Complexity  Matrix 

Let  Qi  be  a cell  equilibrium  point  associated  with  Ia  = Ia(Qi)-  Let  Va(Qi)  (ob- 
tained from  Eqs.(20))  be  the  corresponding  cell  equilibrium  point  of  the  cell  state 
equations  Eqs.(14)-(15).  Let 


J (Qi)  = 


Aaa  (Qi)  A ab(Qi) 

Aba(Qi)  Abb(Qi) 


(21) 


denote  the  m x m Jacobian  matrix  associated  with  fa(Va,  V5)  and  ffc(Va,  V^). 
evaluated  at  (Va(<3i),  Vh(<3i)),  where  'Vb(Qi)  = ga(Va)  from  Eq.(19),  namely, 


Small  Signal  CNN  Cell 
Coefficients  at  Qi 


Let  N(Qi)  be  the  linearized  CNN  cell  at  Qi  associated  with  the  state  Equations 
(14)-(15)  of  the  m-port  cell  C(j,  k.  1)  obtained  by  deleting  the  higher  order  terms 
in  the  Taylor  series  expansion  of  fa(Va,  Vb)  and  fb(Va,  V{,)  about  Qp,  namely, 


Linearized  CNN  Cell 

V„  = AaaVa  + AabVb  + ia 

(23) 

dynamics  at  Qi 

v6  = Afcava  + Abb  Vi, 

(24) 

where  va  = Va  — Va(Qi),  vb  = Vh  — Vb(Qi),  ia  = Ia  - Ia(Qi)  are  the  infinitesimal 
voltages  and  currents,  respectively,  about  the  equilibrium  point  Qi. 
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Definition  1:  Local  Activity 

A CNN  cell  C(j,  k,  l)  is  said  to  be  locally  active  at  Qi  iff  there  exists  some 
\a(t)  and  some  time  T > 0 such  that 

S£(Qi)  = f < va(t),  i a(t)  > dt<  0 (25) 

Jo 

where  < •,  • > denotes  the  vector  dot  product,  and  va(t)  is  the  solution 
obtained  by  solving  Eqs.(23)-(24)  under  zero  initial  state  va(0)  = 0 and 
Vfe(O)  = 0- 

To  derive  a test  for  local  activity,  let  us  take  the  Laplace  transform  of  Eq.(24)  to 
obtain 


sva(s)  = A Mva(s)  + Aafcvb(s)  + ia(«) 


(26) 


svb(s)  = A baVa(s)  + AbbVb(s) 


(27) 


where  va(s),  vb(s)  and  ia(s)  denote  the  Laplace  transform  of  va(t),  vb(f),  and 
ia(t),  respectively.  Solving  for  vb(s)  from  Eq.(27),  we  obtain 


Vfc(s)  = (si  - Abb)  1Abova(s) 


(28) 


Substituting  Eq.(28)  for  vb(s)  in  Eq.(26)  and  solving  for  ia(s),  we  obtain 


ia(s)  = YQ(s)vtt(s) 


(29) 


where 


CNN  Cell  Complex- 
ity Matrix  at  Qi 


is  called  the  CNN  cell  complexity  matrix  at  the  cell  equilibrium  point  Qi.  It 
follows  from  a classic  theorem  in  circuit  theory  33  that  for  a reaction  diffusion 
CNN  Cell  to  be  locally  active  at  Qi,  Y q(s)  should  not  be  a positive  real  matrix  at 
Qi . Hence,  in  order  for  a reaction  diffusion  CNN  equation  to  exhibit  complexity, 
it  is  necessary  that  the  cell  parameters  be  chosen  such  that  the  cell  complexity 
matrix  Y q(s)  is  not  positive  real  at  Qi . The  mathematical  conditions  for  testing 
this  local  activity  property  is  as  follows: 


Yq(s)  = (si  - Aaa)  - Aab(s  1 - Abb)-1  Aba  (30) 
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Local  activity  criteria  for  reaction  diffusion  CNN 

A Reaction  Diffusion  CNN  Cell  is  locally  active  at  a cell  equilibrium  point  Qi 
if,  and  only  if  its  cell  complexity  matrixY q(s),  or  its  inverse  cell  complexity 

matrix  Z q(s)  = Yg1(s)  (In  this  case,  simply  change  the  symbol  from  Y 
to  Z in  the  following  4 conditions)  satisfies  any  one  of  the  following  4 
conditions: 

1.  Y q(s)  has  a pole  in  Re[s]  > 0. 

2.  Y g(zw)  = Yg(icj)  + Yq(ilo)  is  not  a positive  semi-definite  matrix  at 
some  uj  = coo,  where  ojq  is  any  real  number,  and  j denotes  the  Hermitian 
operator. 

3.  Y q(s)  has  a simple  pole  s — iwP  on  the  imaginary  axis  where  its 
associated  residue  matrix 

A lims_iWp(s  - iwp)YQ(s),  if  up  < oo  1 
-1  “ lim^p-too  if  up  = oo  J 

is  either  not  a Hermitian  matrix,  or  else  not  a positive  semi-definite  Her- 
mitian matrix. 

4.  Yq(s)  has  a multiple  pole  on  the  imaginary  axis. 


4-3  Reaction  Diffusion  CNN:  One  Diffusion  Coefficient 

Many  well-known  reaction  diffusion  CNNs  have  only  one  non-zero  diffusion  coeffi- 
cient and  2 state  variables,  i.e.,  m = 1,  n = 2.  The  most  famous  example  belonging 
to  this  class  is  the  FitzHugh  Nagumo  CNN  Equation  given  in  Table  1.  For  this  class 
of  reaction  diffusion  CNN,  all  variables  in  Eq.(24)  are  scalars  and  let  us  rewrite  it 
using  the  established  notation  22  for  ease  of  reference: 

Vi  = auv  i + ai2v2  + ii 

V 2 = O21U1  + 022^2  (31) 

The  inverse  CNN  cell  complexity  matrix  Yff1(s)  ==  Zq(s)  = associated  with 
Eq.(31)  is  a 1 x 1 matrix,  or  scalar  function  of  s in  this  case,  and  is  given  by 


Zq(s) 


( s - (I22) 
s2  — Ts  + A 


(32) 


where 


T — an  + 022 
A = aua22  - 012021 

are  the  trace  and  determinant  of  the  associated  Jacobian  matrix 

3{Qi)  = 

evaluated  at  the  cell  equilibrium  point  Qi 


°n  a 12 
G 21  «22 


(33) 

(34) 

(35) 
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Applying  the  above  local  activity  criteria  to  this  special  scalar  case  (with  Yq(s) 
replaced  by  Zq(s)),  we  obtain  the  following  useful  corollary: 

Local  Activity  Criteria  for  One  Diffusion  Coefficient  A one-port  Reaction-Diffusion 
CNN  cell  with  one  diffusion  coefficient  and  two  state  variables  is  locally  active  at 
a cell  equilibrium  point  Q = (Vi,  Jj)  if,  and  only  if,  any  one  of  the  following  4 
conditions  holds  at  Q: 


1.  an  + a22  > 0 or  ana22  < oi2a2i- 

2.  an  > 0,  or  an  > ai^°221 , if  an  < 0 and  a22  0. 

3.  ana22  > Ui2a2ii  fln  + 022  = 0 and  a22  7^  0. 

4.  ana22  = 012O21,  an  + 022  = 0:  and  a22  7^  0. 

(36) 

Although  the  above  local  activity  criteria  is  couched  in  terms  of  the  4 Jacobian 
coefficients  an,  aj 2,  021  and  a22,  the  criteria  can  be  recast  into  the  following  4 
equivalent  conditions  involving  only  the  3 parameters  A,  T,  and  022- 


Equivalent  condition  1:  T > 0,  or 
A <0 

Equivalent  condition  2:  T > a2 2,  or 

T < a22  and  a22A  > 0 

Equivalent  condition  3:  T = 0,  and  (37) 

A > 0 and 
«22  7 ^ 0 

Equivalent  condition  4:  T = 0,  and 
A = 0 and 
a22  = 0 


To  visualize  the  regions  in  the  A-T-a22  Euclidean  space  represented  by  the  above 
system  of  inequalities,  it  is  more  convenient  to  consider  a A-T-a22  cylindrical  sub- 
space and  partition  it  into  8 uniform  wedges  above  A = 0,  and  8 uniform  wedges 
below  A = 0,  as  depicted  in  Fig. 9.  In  terms  of  the  A-T-a22  cylinder,  Equivalent 
condition  1 is  represented  by  all  points  (022,  T,  A)  behind  the  vertical  separating 
plane  T = 0,  and  all  points  below  the  horizontal  separating  plane  A = 0.  Equiva- 
lent condition  2 is  represented  by  the  wedges  labeled  [T],  [2],  | 3 |,  |~4~|,  [~5~|,  [~7|,  and 
[~8]  in  the  upper  half  (A  > 0) 
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Figure  7.  A 2-dimensional  reaction-diffusion  CNN  having  “m”  non-zero  diffusion  coefficients 
( Di  -/  0 , i = l,2,...,rn).  The  subscript  “i”  attached  to  ( ) denotes  the  node  in  the  ith  re- 
sistive grid  layer  which  is  connected  to  a terminal  of  cell  C(j,  k).  All  resistors  in  layer  i are  linear 
resistors  with  identical  conductance  equal  to  Di  Siemens,  i = 1, 2, m. 
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cylinder.  Equivalent  condition  3 is  represented  by  the  vertical  separating 
plane(excluding  the  vertical  axis)  in  the  upper  half  (A  > 0)  cylinder.  Equiva- 
lent condition  4 is  represented  by  a single  point  at  the  origin.  It  follows  from  this 
partitioned  cylinder  that  a reaction  diffusion  CNN  with  one  diffusion  coefficient  and 
two  state  variables  is  locally  active  at  a cell  equilibrium  point  Q if,  and  only  if,  its 
associated  parameter  (a22(Q),T(Q),  A(Q))  at  Q lies  outside  of  the  “blue”  sector 
|~6~|  in  the  upper  half  cylinder. 

4-4  Reaction  Diffusion  CNN:  Two  Diffusion  Coefficients 

Consider  next  the  class  of  reaction  diffusion  CNNs  with  two  diffusion  coefficients 
(Di  > 0 and  D2  > 0)  and  two  state  variables,  i.e. , m = n = 2.  Both  the  Brusselator 
and  the  Meinhardt-Gierer  CNN  equations  in  Table  1 belong  to  this  class.  In  this 
case,  the  CNN  cell  is  a 3- terminal  (including  the  ground  reference  terminal),  or 
2-port , device  whose  linearized  cell  dynamics  about  an  equilibrium  point  Qi  is 
described  by 

v 1 — anv!  + ai2V2  + ii 

v2  = a2\V\  + a22V2  + h (38) 

where  an,  a\2,  021,  and  022  are  the  small-signal  cell  coefficients  at  Qi  defined  in 
Eq.(35).  Applying  the  local  activity  criteria  from  section  4.2  to  the  associated  2x2 
cell  complexity  matrix  Y q(s)  at  Qi,  we  obtain  the  following  corollary: 

Local  activity  criteria  for  two  diffusion  coefficients 

A two-port  reaction-diffusion  CNN  cell  with  two  diffusion  coefficients  and  two 
state  variables  is  locally  active  at  a cell  equilibrium  point  Q = (V),  V2,  h,  I 2)  if, 
and  only  if,  any  one  of  the  following  two  conditions  holds  at  Q: 


1.  a22  > 0 

2.  4ana22  < (012  + a2i)2 

(39) 

For  a detailed  application  of  the  various  local  activity  criteria  presented  in  Sections 
4.3  and  4.4  to  concrete  examples;  namely,  the  FitzHugh-Nagumo  Equation,  the 
Brusselator  Equation,  and  the  Gierer-Mainhardt  Equation,  the  reader  is  referred 
to  34 , 35 , and  36 , respectively. 

5 Concluding  Remarks 

The  preceding  analysis  can  be  generalized  to  any  homogeneous  media  which  can  be 
mapped  to  a CNN  defined  by  any  cell  dynamics,  and  any  cell  interaction  laws,  not 
necessarily  of  the  reaction  diffusion  type  presented  in  Section  4.  In  particular,  the 
coupling  7-port  Fjki  can  be  any  nonlinear  dynamical  multi-port.  In  such  cases,  in 
order  for  the  CNN  to  exhibit  complexity,  either  the  m-port  cell  Cy*,,  or  the 


Figure  9.  The  A-T-C122  cylinder  depicting  the  locally  passive  region  represented  by  wedge  j 6 | in 
the  upper  half  cylinder.  All  other  regions  are  locally  active.  In  particular,  regions  |jT],  | 7 |,  and 
[~8~j  in  the  upper  half  cylinder  A > 0 correspond  to  the  edge  of  chaos  where  most  complexities 
emerged. 
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coupling  7-port  Fijk,  must  be  locally  active  at  some  equilibrium  point  of  the  isolated 
(disconnected  from  the  rest  of  the  CNN)  cell  Cijk,  or  isolated  7 -port  coupling  Fjki- 
While  the  CNN  paradigm  is  an  example  of  REDUCTIONISM  par  excellence , 
the  true  origin  of  emergence  and  complexity  is  traced  to  a much  deeper  new  con- 
cept called  local  activity.  The  numerous  complex  phenomena  unified  under  this 
mathematically  precise  principle  include  self  organization , dissipative  structures, 
synergetics , order  from  disorder,  far-from-thermodynamic  equilibrium , collective  be- 
haviors, edge  of  chaos,  etc. 

The  central  theme  of  the  local  activity  dogma  22  asserts  that  the  somewhat  fuzzy 
notions  of  “emergence”  and  “complexity” , as  well  as  their  various  metamorphosis, 
such  as  those  cited  above,  can  all  be  rigorously  explained  by  a precise  scientific 
paradigm  abstracted  mathematically  from  the  principle  of  conservation  of  energy; 
namely,  a CAW  operating  near  the  edge  of  chaos  34>37,  where  the  cells  are  not  only 
locally  active,  but  also  linearly  asymptotically  stable.  In  particular,  constructive 
and  explicit  mathematical  inequalities  are  given  for  identifying  the  region  in  the 
CNN  parameter  space  where  complex  phenomena  may  emerge,  as  well  as  for 
localizing  it  further  into  a relatively  small  parameter  domain  called  the  edge  of 
chaos  (regions  [5],  [T],  and  jjT|  in  Fig. 9)  where  the  potential  for  emergence  is 
maximized. 

Consequently:  he  who  wants  to  have  right  without  wrong, 
Order  without  disorder, 

Does  not  understand  the  principles 
Of  heaven  and  earth. 

He  does  not  know  how 
Things  hang  together. 

Chuang  Tzu 
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